Education in the Knowledge Society 23 (2022)

Can Gamification Help in Increasing Motivation, Engagement, and Satisfaction? A gamified experience in teaching CS to students from other disciplines

¿Puede la gamificación ayudar a aumentar la motivación, el compromiso y la satisfacción? Experiencia gamificada en la enseñanza de la informática en alumnos de otras disciplinas

Miguel García-Iruela^a(*), Raquel Hijón-Neira^b, Cornelia Connolly^c

^a Computer Science Department, Universidad Rey Juan Carlos, Madrid, España.

https://orcid.org/0000-0003-2383-6401 miguel.garciai@urjc.es.

^b Computer Science Department, Universidad Rey Juan Carlos, Madrid, España.

https://orcid.org/0000-0003-3833-4228 raquel.hijon@urjc.es

^c School of Education, College of Arts, Social Sciences, & Celtic Studies, National University of Ireland Galway, Ireland. Lero, SFI Centre for Software Research, National University of Ireland Galway, Ireland.

https://orcid.org/0000-0001-9778-5569 cornelia.connolly@nuigalway.ie

^* Corresponding author

ABSTRACT

The digital revolution is causing the new generations to demand new ways of learning. Motivating students is one crucial element of the learning-teaching process to achieve students’ learning, which is the main objective of education. Using typical gaming elements in non-game environments, gamification transforms the learning process into a more motivating and enjoyable experience. Therefore, gamification offers adequate solutions for the educational needs of the new generation of students. Although most studies have found benefits from gamification, the results have not been encouraging in some cases. Several review studies suggest that more empirical studies are needed to investigate the motivating effects of using gamification in different educational settings and for specific student types. This study wants to answer whether using the proposed gamification design affects motivation, engagement, and satisfaction in teaching computer science to university students from other disciplinary areas such as journalism and audiovisual communication. The experiment consists of a gamified experience in a virtual classroom in Moodle for four weeks on general computer science content. A control group and a test group were used. The difference between the two was the use of gamified elements (points, badges, levels, task unlocking...). From the data obtained in this study, no significant differences were found in motivation and satisfaction between the different groups or the different methodologies. Our results demonstrate a specific positive correlation between engagement, motivation, and satisfaction. It is worth noting the positive values obtained in this type of course among all the students who were part of this experience and methodology.

Keywords
gaming; computer science education; learning sciences; motivation; engagement; satisfaction.

RESUMEN

La revolución digital está provocando que las nuevas generaciones demanden nuevas formas de aprendizaje. Motivar a los estudiantes a que presten atención y se comprometan con el material se ha convertido en uno de los objetivos de la educación. La gamificación, mediante el uso de elementos típicos del juego en entornos ajenos al juego, transforma el proceso de aprendizaje en una experiencia más motivadora y placentera, por lo que la gamificación ofrece soluciones adecuadas a las necesidades educativas de la nueva generación de estudiantes. Pese a que la mayoría de los estudios han encontrado beneficios del uso de la gamificación, en algunos casos los resultados no han sido alentadores. Varios estudios de revisión sugieren que se necesitan más estudios empíricos para investigar los efectos motivadores del uso de la gamificación en diferentes entornos educativos y para tipos específicos de estudiantes. Este estudio quiere dar respuesta a si el uso del diseño de gamificación propuesto afecta a la motivación, el compromiso y la satisfacción en la enseñanza de la informática en estudiantes universitarios de otras áreas disciplinares como el periodismo y la comunicación audiovisual. El experimento consta de una experiencia gamificada en un aula virtual en Moodle durante cuatro semanas sobre contenidos generales de informática. Se empleó un grupo de control y un grupo test, la diferencia entre ambos era el empleo de elementos gamificados (puntos, insignias, niveles, desbloqueo de tareas…). A partir de los datos obtenidos en este estudio no se encontraron diferencias significativas en motivación y satisfacción entre los distintos grupos ni entre las distintas metodologías. Nuestros resultados demuestran una cierta correlación positiva entre compromiso, motivación y satisfacción. Cabe destacar los valores positivos obtenidos en este tipo de cursos entre todos los alumnos que formaron parte de esta experiencia y metodología.

Palabras clave
juego; educación en informática; ciencias del aprendizaje; motivación; compromiso; satisfacción.

1. Introduction

The digital revolution is rapidly changing the world and humanity. Younger generations, who are born with digital technologies, called Millennials or Y-Generation, grow up in a world where everyone has a computer in their pocket (Koivisto & Hamari, 2014). These modern children prefer to learn information that is useful, fun, and relevant (Jukes & Dosaj, 2004), they learn differently (Prensky, 2014). The requirement to know how this generation can learn better and what their preferred learning styles are (Arabaci & Polat, 2013) has become a new educational problem (Campbell, 2016).

Motivating students to pay attention and engage to the material has become one of the central objectives of education. More importantly, educators want the desire to learn to come from the student and therefore the students to be intrinsically motivated to learn (Deci & Ryan, 2000). It is in this way that the use of gamification can be useful.

1.1 Gamification

Gamification refers to the design of systems, services, and processes to provide positive and engaging experiences like the engaging experiences provided by games, commonly with the aim of motivating beneficial behaviors (Hamari, 2019; Landers et al., 2018). The scientific definition of gamification is defined as the process of applying elements of the game to contexts outside the game (Ding et al., 2018; Domínguez et al., 2013; Schöbel et al., 2020; Zimmerling et al., 2019). The elements of the game most adopted in various fields of study are levels, points, badges, leaderboards, and avatars (Barata et al., 2017). There are many other mechanisms that are also available in gamified systems such as combat, unlock content, boss fights, quests, social graphics, certificates, and memes (Buckley & Doyle, 2017).

Since its conceptual beginning around 2010, gamification has increasingly caught the attention of academics and professionals (Hamari et al., 2014). In addition to winning popular advocates, the approach has gained momentum from the positive outlooks published in the Gartner (2011) and IEEE (2014) business analysis that predict that most companies and organizations will implement gamification in the near future. Gamification has been considered one of the emerging and widely adopted key teaching technologies in education in this decade (Johnson, L. et al., 2016). Gamification offers adequate solutions for the educational needs of the new generation of students (Bíró, 2014).

1.2 Motivation, engagement, and satisfaction

The notion of gamification makes the course and learning exercise more engaging and encourages students to work more actively through digital means, such as earning points, badges, and earning a top position on a leaderboard (Barata et al., 2017; Baydas & Cicek, 2019). The gamification component is believed to be effective in increasing student motivation and improving their learning experience, engagement, and performance. From a pedagogical perspective, it is believed that the provision of extrinsic rewards is likely to harm intrinsic motivation (Mekler et al., 2017). Conversely, some authors have stated that gamification influences extrinsic and intrinsic motivation (Adukaite et al., 2017; Jurgelaitis et al., 2019).

The values of commitment and motivation always correlate positively with the academic performance of a student. The more committed a student is, the more likely they are to participate and the more intrinsic motivation they have (Coffman, 2012). Additionally, along with motivation and engagement, the inclusion of student satisfaction is considered in this study.

2. Previous Studies

Gamification approaches are being applied more frequently to positively affect behavior and cognitive processes by improving the system or service with possibilities of motivation and, finally, providing experiences like those of games (Huotari & Hamari, 2017).

Motivational possibilities have been widely used in many fields, such as business (Alcivar & Abad, 2016; Xi & Hamari, 2020), logistics (Warmelink et al., 2020), crowdsourcing (Morschheuser et al., 2017; Mason et al., 2012), retail (Poncin et al., 2017), innovation (Leclercq et al., 2017), healthcare (Hammedi et al., 2017; Johnson, D. et al., 2016; Cafazzo et al., 2012), banking (Rodrigues et al., 2016), mobile marketing (Hofacker et al., 2016; Souiden et al., 2019), human resource management (Kim, 2018) and education (de Pontes et al., 2019; Dichev & Dicheva, 2017; Dicheva et al., 2019; Fraser et al., 2019; Hanus, & Fox, 2015; Koivisto, & Hamari, 2019; Majuri et al., 2018; Osatuyi et al., 2018; Seaborn & Fels, 2015). Furthermore, gamification has been used in many contexts related to education, at different educational levels: from primary school to higher education (Caponetto et al., 2014); higher education, training and tutorials, languages, elementary education, and lifelong education (de Sousa Borges et al., 2014); higher education and training Dicheva et al., 2015); K-6 level (Simões et al., 2013). Also, it has been used in various subjects, such as computer science, math, multimedia, communication, medicine, biology, psychology (Dicheva et al., 2015) or online communities and social networks, health and wellness, crowdsourcing, sustainability, orientation; research, marketing (Seaborn & Fels, 2015). All these options show their potential to improve learning outcomes (Koivisto & Hamari, 2019; Seaborn & Fels, 2015).

The main objective of gamification is to increase engagement (Kapp, 2012; Villagrasa et al., 2014). Low student participation and lack of motivation are the main problems that teachers must address in schools (Lee & Hammer, 2011). For this reason, gamification has been applied mainly in educational contexts (De-Marcos et al., 2017; Hamari et al., 2014).

Although several empirical studies have alluded to the positive impact of gamified learning results (Göksün & Gürsoy, 2019; Hassan et al., 2019; Huang & Hew, 2018; Huang et al., 2019; Lo & Hew, 2020), other studies have produced contradictory results (Baydas & Cicek, 2019; Ding, 2019; Kyewski & Krämer, 2018; Rachels & Rockinson-Szapkiw, 2018). Some studies stated that the use of points, badges, levels, and leaderboards did not promote students' sense of community and did not increase students' competence, their need for satisfaction and intrinsic motivation (Ding et al., 2017; Mekler et al., 2017). For example, the study by Kyewski and Kramer (2018) reported that insignia did not successfully increase intrinsic motivation during the instructional period. These findings also suggest that gamification is not always appropriate for all types of content or context.

Review studies on the effectiveness of gamification are generally optimistic, mainly listing positive or mixed results of applied gamified strategies (Buckley & Doyle, 2017; Caponetto et al., 2014; Dicheva et al., 2015; Koivisto & Hamari, 2019; Lambruschini, & Pizarro, 2015; Majuri et al., 2018; Nah et al, 2014; Osatuyi et al., 2018; Reiners et al., 2012; Seaborn & Fels, 2015). However, they mention the need for more controlled experimental research on the impact of gamification, regardless of the application domain or the gamified strategy used (Buckley & Doyle, 2017; Caponetto et al., 2014; Dichev & Dicheva, 2017; Dicheva et al., 2015; Koivisto & Hamari, 2019; Majuri et al., 2018; Lambruschini, & Pizarro, 2015; Landers et al., 2018; Nah et al, 2014; Osatuyi et al., 2018; Reiners et al., 2012; Seaborn & Fels, 2015).

Dicheva et al. (2015) conducted a mapping study of gamification in education that investigated current empirical research on its implementation. They found that the reviewed articles reported promising results, but most empirical studies did not provide a rigorous evaluation, making it difficult to understand the reasons behind the positive or negative results. Therefore, Dicheva et al. (2015) suggest that more empirical studies are needed to investigate the motivating effects of using game elements in different educational contexts and for specific types of students. Although gamification has an important position in education, both within and outside universities, there is still little effective guidance on how to combine different gamification functions to improve learning performance in different educational contexts (Hanus & Fox, 2015; Koivisto & Hamari, 2019; Seaborn & Fels, 2015).

This study presents a gamified experience that brings together the main elements of gamification to observe the effect that the proposed design has on motivation, engagement and satisfaction in learning general computer concepts in university students belonging to the journalism and audiovisual communication area. The question we want to answer is whether the use of gamification affects motivation, engagement, and satisfaction in teaching CS in students from other disciplines.

3. Research Method

In March of 2019 in the public University Rey Juan Carlos, as part of the laboratory of the subject New Technologies in Information Systems it was proposed to carry out a series of activities in a course implemented under the Moodle platform, to check if it is true that the gamification in this environment can have a positive effect on motivation, engagement, and satisfaction.

The duration of the laboratory was 4 weeks, the same as the proposed course. The course was about general concepts in computing: hardware, software, operating systems, networks, security, and licenses. The subject is integrated into the university grades to provide students with basic technological knowledge to students in the area of humanities and social sciences.

3.1. Sample

The average age of the students was 20 years old. They are studying the first year of university in different degrees, but all of them grouped in the same branch of knowledge. Unlike previous experiences (Garcia-Iruela, et al., 2020) this group of students did not belong to a technical profile. By belonging to different cycles and schedule shifts, the students separated into 6 different groups as we can see in table 1. The decision did not consider age or previous experience, this group was chosen so that they knew the classmates with whom they shared a course and did not detect that some were an experimental group and others a control group. Each group in the experiment corresponds to a group with a different time shift at the university.

Half of the groups were in a gamified course, in total there were 174 registered students. The other half was a control group, in total there were 171 students enrolled in the course. Of the total number of registered students, 146 reached the final in the experimental groups and 150 in the control groups. There were 21 dropouts (12.1%) among control students and 28 among experimental students (16.4%), in total 14.2% of students dropped out. In the analysis of the experience, only the sample of students who completed it was considered, as there were no data in the surveys of the students who dropped out.

To sum up the sample information, in Table 1 we can observe by group if it was gamified, the degree, the number of students, the numbers of women, the number of men, the number of dropouts, and the teacher of the group.

3.2. Research design

The experiment is carried out for a month with the students of the face-to-face subject of new technologies from journalism and audiovisual communication. As part of the subject's laboratory, a virtual classroom was created in Moodle with additional activities that were worth 10% of the laboratory grade. This virtual classroom was accessible from the university and from home, the questionnaires and Moodle tasks was designed to be done from home, but some students completed them in-class sessions without teacher guidance. There were no restrictions to avoid this.

The subject has 6 ECTS credits and consisted of two face-to-face sessions of two hours per week, the proposed experience required an additional load of approximately half an hour per week.

The study used an experimental design between subjects in which the students were randomized into six groups (3 experimental groups and 3 control groups). The characteristics of the students such as age, sex or knowledge of the subject were not used for the distribution. Both groups had 4 weeks with the same activities to do. The only difference is that the experimental group enjoyed a gamified course in which they obtained points, badges, unlock tasks... while the control group did not have the gamified elements. The gamification elements used were:

The choice of elements is due to their widespread use and the possibility of integrating them all into the Moodle platform using the Gamemo plugin (García-Iruela & Hijón-Neira, 2018).

3.3. Surveys

To obtain feedback on the motivation and engagement of the students, it was decided to carry out a survey of the students at the end of the course. The survey has three different parts, the first to measure motivation, the second to measure engagement, and the third to measure satisfaction. In this way we can measure those aspects of each student at the end of the experience. Each of these parts is inspired by previously tested questionnaires specified in this section.

3.3.1. Motivation

The first part of the survey has used the Intrinsic Motivation Inventory (IMI), “It is a multidimensional measurement device intended to assess participants subjective experience related to a target activity in laboratory experiments” (Ryan & Deci, 2006). This allows us to evaluate different scales related to motivation, depending on what you want to measure, one or the other subscales will be used without affecting the results (Ryan & Deci, 2006). In our study, we considered the following scales important for evaluating motivation: “Interest/enjoyment”, “Perceived competence”, “Perceived choice”, “Pressure/tension” and “Effort/importance”.

At least 4 questions have been used to measure each scale since the incremental R for each element above 4 items for any given factor are quite small (Ryan & Deci, 2006). Each question had to be answered using a Likert scale from 1 (not all true) to 7 (very true).

3.3.2. Satisfaction

This part of the survey has seven questions common to the experimental and control groups, the first 4 questions have been collected from the satisfaction section of the USE questionnaire (Lund, 2001) and the other three from (De-Marcos et al., 2014). All these questions are rated from 1 (Strongly disagree) to 7 (Strongly agree).

3.3.3. Engagement

This part of the survey is based on the short version of the user engagement scale (UES) developed in “A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form” (O’Brien et al., 2018). The engagement measure is based on four factors:

They have been formulated in a random order, preventing students from detecting the factor to be measured, in addition to adapting the questions to the context as recommended in (O’Brien et al., 2018). Each factor integrates three questions using a Likert scale from 1 to 5.

4. Analysis

To analyse the data, we will first show the global and sex data of gamified students versus non-gamified students, considering motivation, satisfaction and engagement. The information will then be displayed for each of the 6 groups (3 gamified and 3 control).

4.1 Gamified vs Control

4.1.1. Motivation and satisfaction

After 4 weeks, at the end of the course, students completed a survey to measure motivation, satisfaction and engagement. Table 2 shows the mean, maximum value, minimum value, and standard deviation of the gamified and control group obtained in motivation and satisfaction global and by sex.